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Abstract 

The paper is focused on the tradeoff between performance and decoding complexity per iteration for LDPC codes 
in terms of their gap (in rate) to capacity. The study of this tradeoff is done via information-theoretic bounds 
which also enable to get an indication on the sub-optimality of message-passing iterative decoding algorithms (as 
compared to optimal ML decoding). The bounds are generalized for parallel channels, and are applied to ensembles 
of punctured LDPC codes where both intentional and random puncturing are addressed. This work suggests an 
improvement in the tightness of some information-theoretic bounds which were previously derived by Burshtein et 
al. and by Sason and Urbanke. 


1 Introduction 

Error-correcting codes which employ iterative decoding 
algorithms are widely considered state of the art in 
channel coding. Their outstanding performance and 
feasible encoding and decoding complexities motivate 
a theoretical study of the tradeoff between their perfor¬ 
mance and complexity. While it would be very inter¬ 
esting to explore this tradeoff for finite block lengths, 
this central issue is still beyond the scope of rigorous 
analysis. Via information-theoretic bounds, we study 
this tradeoff in the asymptotic case where the block 
length tends to infinity. This study is focused on low- 
density parity-check (LDPC) codes, but we also address 
other constructions of codes on graphs which closely 
approach channel capacity with moderate complexity. 

LDPC codes are efficiently encoded and decoded due 
to the sparseness of their parity-check matrices ([18], 
[19], [20]). An ensemble of LDPC codes is defined by 
a pair of degree distributions A(a;) = 2 A iX l ~ x and 

p(x) = X^i >2 Pi x% ~ X where \ (pi) is the fraction of 
edges in the bipartite graph adjacent to variable (parity- 
check) nodes of degree i. Consider the number of ones 
in a parity-check matrix which represents a binary 
linear block code, and normalize it per information 
bit (i.e., with respect to the dimension of the code). 
This quantity (defined as the parity-check density [21]) 
is linearly proportional to the number of messages 
produced in each iteration of the message-passing it¬ 
erative (MPI) decoder, normalized per information bit. 
Therefore, the density of the parity-check matrix can be 
regarded as a measure for the decoding complexity per 
iteration of LDPC codes normalized per information 
bit. Our study of the tradeoff between performance and 
complexity per iteration is initiated by two questions 
which were addressed by Burshtein et al. [2] and Sason 
and Urbanke [21]: 


Question 1: How sparse can parity-check matrices 
of binary linear codes be, as a function of the gap 
between the code rate for which reliable communication 
can be achieved and the channel capacity? 

Question 2: How good can LDPC codes be (even 
under maximum-likelihood (ML) decoding), as a func¬ 
tion of their degree distributions? 

The answer to the first question was given in [21] in 
the form of lower bounds on the parity-check density 
(which serve as lower bounds on the decoding com¬ 
plexity per iteration); these lower bounds were later 
improved by the authors [26], and the improved bounds 
will be introduced in this paper. Since the density 
of a parity-check matrix is related to the number of 
fundamental cycles in the corresponding bipartite graph 
(see [21]), this also gives a quantitative measure to 
the statement that bipartite graphs representing good 
error-correcting codes should have cycles [3]. Referring 
to the second question, the answer is given in the 
form of upper bounds on the achievable rates whose 
derivation is based on performance bounds under ML 
decoding. Comparing these upper bounds with exact 
thresholds under iterative decoding, calculated using 
density evolution (DE) [19], leads to a quantitative 
measure of the inherent loss in performance due to 
the sub-optimality of MPI decoding. The derivation of 
these bounds also allows to derive lower bounds on 
the bit/block-error probability under ML decoding (see 
[21], [26]). 

In his thesis [4, Theorem 3.3], Gallager provided 
an upper bound on the rate of right-regular LDPC 
codes (i.e., LDPC codes with a constant degree (or) 
of the parity-check nodes) which is a necessary con¬ 
dition for achieving reliable communications over a 
binary symmetric channel (BSC), even under optimal 
ML decoding. This bound shows that right-regular 
LDPC codes with fixed right degree cannot achieve 


the channel capacity on a BSC, even under optimal 
ML decoding. The inherent gap between the achievable 
rates and the channel capacity is well approximated by 
an expression which decreases to zero exponentially 
fast in or. Burshtein et al. have generalized Gallager’s 
bound for general ensembles of LDPC codes trans¬ 
mitted over memoryless binary-input output-symmetric 
(MBIOS) channels [2]; to this end, they applied a two- 
level quantization to the log-likelihood ratio (LLR) of 
these channels which essentially turns them into a BSC. 

In [ 8 ], Khandekar and McEliece have suggested 
to study the encoding and decoding complexities of 
ensembles of iteratively decoded codes on graphs as a 
function of their gap to capacity. They conjectured that 
if the achievable rate under MPI decoding is a fraction 
1 — £ of the channel capacity, then for a wide class of 
channels, the number of iterations required for decoding 
with vanishing bit error probability scales like - and the 
density of the parity-check matrix scales like In j. We 
note that recent findings on the matching condition for 
GEXIT curves of capacity-approaching codes seem to 
suggest an analytical tool for proving the conjecture that 
the average number of iterations scales like the inverse 
of the multiplicative gap (s) to capacity [12], Based 
on this conjecture, it follows that for irregular repeat- 
accumulate codes, the encoding complexity scales like 
Ini anc j the decoding complexity scales like 7 In 7 . 
The only exception is the binary erasure channel (BEC) 
where the decoding complexity behaves like In 7 (same 
as encoding complexity) because of the absolute relia¬ 
bility of the messages passed through the edges of the 
graph (hence, every edge can be used only once during 
the iterative decoding process). 

Based on the work in [2], Sason and Urbanke derived 
an information-theoretic lower bound on the asymptotic 
density of parity-check matrices [21, Theorem 2.1] 
where this bound applies to every MBIOS channel and 
every sequence of binary linear block codes achieving a 
fraction 1 — e of the channel capacity with vanishing bit 
error probability. It holds for an arbitrary representation 
of full-rank parity-check matrices for these codes, and 
is of the form — -- where K\ and A '2 are 
constants which only depend on the channel. For the 
BEC they used considerations relating to the rank of the 
parity-check matrix to get an improved bound, which 
is also logarithmic in 7. A natural question arises: 

Question 3: Is the logarithmic behavior of the lower 
bound on the parity-check density true, or is it just an 
artifact of the bounding technique ? 

Sason and Urbanke showed that for any MBIOS chan¬ 
nel, the logarithmic behavior of the parity-check density 
is achievable under ML decoding [21, Theorem 2.2], 
A recent analysis in this direction was also presented 
in [ 6 ]. For the BEC, they showed that the sequence of 
right-regular ensembles of Shokrollahi [24] achieves the 
improved bound (up to a small additive constant) even 
under (sub-optimal) MPI decoding [21, Theorem 2.3]. 


In [10], Measson et al. derived an upper bound 
on the thresholds of ensembles of LDPC codes un¬ 
der maximum a posteriori (MAP) decoding where the 
transmission of these codes takes place over the BEC. 
Their general approach relies on extrinsic information 
transfer (EXIT) charts. Generalized extrinsic informa¬ 
tion transfer (GEXIT) charts were recently introduced 
by Measson et al. ([9], [12]); they form an extension 
of EXIT charts to general MBIOS channels, and they 
fulfill the area theorem. This preservation of the area 
theorem for general MBIOS channels enables to get 
upper bounds on the thresholds of turbo-like ensembles 
under MAP decoding. Under some conditions, the 
bound in [10] was shown to be tight for the BEC. 

A new method for analyzing LDPC codes and low- 
density generator-matrix (LDGM) codes under bit- 
MAP decoding is introduced by Montanari in [11]. 
The method is based on a rigorous approach to spin 
glasses, and allows a derivation of an upper bound 
on the achievable rates of these codes under bit-MAP 
decoding. The computational complexity of this bound 
grows exponentially with the maximal right and left 
degrees (see [11, Eqs. (6.2) and (6.3)]). Since the 
bounds in [ 10 ], [ 11 ] are derived for ensembles of 
codes, they are probabilistic in their nature; based on 
concentration arguments, they hold asymptotically in 
probability 1 as the block length goes to infinity. It 
was conjectured that the bounds in [ 11 ], which hold 
for general LDPC and LDGM ensembles over MBIOS 
channels, are tight. 

In [1] and [25], Ardakani et al. suggest an ap¬ 
proximated one-dimensional analysis which enables 
to optimize (numerically) the degree distributions for 
achieving a good tradeoff between performance and 
complexity. 

The derivation of the results presented in this paper 
was motivated by the fact that the information-theoretic 
bounds in [ 2 ] and [ 21 ] rely on a two-level quantization 
of the LLR and do not take into consideration the entire 
statistics of the communications channel. In this paper 
we present a new information-theoretic upper bound on 
the achievable rates of LDPC codes transmitted over 
MBIOS channels. This bound holds for optimal ML 
decoding and is therefore valid under any sub-optimal 
decoding algorithm. We also discuss an improvement 
of the information-theoretic lower bound on the parity- 
check density. These bounds hold for every sequence of 
codes and do not rely on probabilistic arguments. The 
bounds are generalized for the case where the codes are 
transmitted over a set of independent parallel MBIOS 
channels, where each code bit is a-priory assigned to 
one of the parallel channels. This setup can be used to 
model different scenarios which include non-uniform 
error protection [16], punctured LDPC codes [5], [17] 
and LDPC-coded modulation schemes. The bounds for 
parallel channels are used to bound the performance 
and the decoding complexity per iteration of punctured 



LDPC codes transmitted over a single MBIOS channel. 

Due to space limitations, the proofs of the statements 
in this paper are relegated to the full paper versions 
(see [23] and [26]); further discussion on the relations 
between these theorems and some previously reported 
results from [2], [14], [16], [21] are presented there. 


2 Results for an MBIOS Channel 

In this section we present new upper bounds on the 
achievable rates of LDPC codes, and new lower bounds 
on their bit-error probability. The transmission of the 
codes is assumed to take place over an MBIOS channel 
and the bounds refer to ML decoding. We also consider 
an improved lower bound on the parity-check density 
of a sequence of LDPC codes which achieve a fraction 
1 — e of the capacity of an MBIOS channel with 
vanishing bit error probability. These bounds tighten the 
results in [2], [21]. The upper bound on the achievable 
rates and the lower bound on the parity-check density 
in [ 2 ], [ 21 ] are both derived from a lower bound of 
the conditional entropy of the transmitted codeword 
given the received sequence at the output of the com¬ 
munication channel. For the derivation of this bound 
Burshtein et al. considered the entropy of the syndrome 
of the received sequence, this requires that the received 
sequence be binary. Hence, a two level quantization 
which essentially turns the MBIOS channel into the 
BSC was applied to the LLR at the output of the 
communication channel. This quantization causes an 
inherent loss in the tightness of the bounds in [ 2 ], [ 21 ]. 

We introduce two approaches to circumvent the 
two-level quantization of the LLR. The first approach 
forms a generalization of the approach introduced by 
Bursthein et al. and allows a finer quantization of 
the LLR to a number of levels which is an arbitrary 
integer power of 2. Each quantization level is mapped 
to a member of the Galois field GF(2 d ) (where the 
number of quantization levels is 2 d and d £ N), and 
the concept of syndrome is generalized for members of 
GF(2 d ). These bounds are subject to optimization of 
the quantization levels, as to achieve the tightest bounds 
within their form. For optimally chosen quantization 
levels it can be shown that the bounds become tighter 
as the number of levels grows. A second approach is 
to consider the soft value of the LLR which forms a 
sufficient statistics of ML decoding. In this approach 
the LLR is split into its sign and its absolute value, 
due to the symmetry of the channel it is possible to 
define the syndrome w.r.t the sign of the LLR and 
use its absolute value as side information. This second 
approach leads to bounds which are uniformly tighter 
than the first approach and require no optimization. 
However, comparing the results of both approaches 
gives insight on the effect of quantization of the LLR 
at the decoder due to implementation restrictions. Due 
to space limitations we only present bounds which are 


based on the second approach. Some numerical results, 
presented at the end of this section, demonstrate the 
results given by both approaches. 

As noted above, the bounds on the achievable rates, 
the bit error probability and the parity-check density 
are derived from a lower bound on the entropy of 
the transmitted codeword conditioned on the received 
sequence. We present an improved lower bound on this 
entropy, as compared to the one derived in [ 2 ]. 

Theorem 2.1: Let C be a binary linear code of length 
n and rate R transmitted over an MBIOS channel. 
Let x = (xi,x 2 , ■ ■ ■ ,x n ) and y = (yi,y 2 , • • • • Vn) be 
the transmitted codeword and the received sequence, 
respectively. For an arbitrary representation of the code 
C by a full-rank parity-check matrix, let F^ be the 
fraction of the parity-check equations of degree k, 
and F (x) = F f .x k designate the corresponding 

right degree distribution of C. Then the conditional 
entropy of the transmitted codeword given the received 
sequence satisfies 


g(X|Y) 

n 


> 1-C-(1-R) 


1 y- r(ffp) \ 

21 n( 2 )^p( 2 p-l)j 


where 

a(l)(l+e~ l ) tanh 2p ^ dl , ( 1 ) 

and a is the conditional pdf of the LLR given the 
channel input is 1 (where the possible inputs to the 
channel are {+ 1 ,- 1 }). 

From Fano’s inequality and since the code is binary, 
the conditional entropy of the transmitted codeword 
given the received sequence satisfies 

iT(XIY) , N 
V 1 ; < Rh P b (2) 

n 

where R is the rate of the code, Pb is the bit-error prob¬ 
ability of the ML decoder, and h{x) = —xlog 2 (x) — 
(1 — x)log 2 (l — x) is the binary entropy function to 
base two. By combining Theorem 12.11 and Eq. 0 , it 
is possible to derive an upper bound on the achievable 
rates of sequences of binary linear block codes and a 
lower bound on their bit-error probability, these bounds 
are presented in the next corollaries. 

Corollary 2.1: Let { C m } be a sequence of binary 
linear block codes whose codewords are transmitted 
with equal probability over an MBIOS channel, and 
assume that the block lengths of these codes tend to 
infinity as m —> oo. Let Tk, m be the fraction of 
the parity-check nodes of degree k for an arbitrary 
representation of the code C m by a bipartite graph 
which corresponds to a full-rank parity-check matrix, 
and assume the limit Fk = lirri„ woc L/ i:m exits. Then 
in the limit where m —> oo, a necessary condition 
on the asymptotic achievable rate (R) for obtaining 







vanishing bit error probability is 

1-C 


R< 1 - 


1 - 


1 


OO 


r (g P ) 


21n(2)^p(2p-l) 

Corollary 2.2: Let C be a binary linear block code of 
rate R whose transmission takes place over an MBIOS 
channel with capacity C. For an arbitrary full-rank 
parity-check matrix H of the code C, let F*. designate 
the fraction of parity-check equations that involve k 
variables, and T(:r) = ^ fc TfcX fc be the right degree 
distribution from the node perspective which refers to 
the corresponding bipartite graph of C. Then, under 
ML decoding (or any other decoding algorithm), the 
bit error probability (Pb) of the code satisfies 


MA) > l - % + 


OO 


r (g P ) 


R 2Pln(2) z ^ i p(2p- 1)' 

By combining Theorem 12. II and Eq. 0, one can also 
derive a lower bound on the asymptotic parity-check 
density of a sequence of LDPC codes which achieve 
a fraction 1 — e with vanishing bit-error probability. 
Similarly to the bound of Sason and Urbanke this bound 
is also of the form —- —, but the new bound 
is tighter then the one given in [21], Particularly, the 
coefficient of the logarithmic growth rate Ki is strictly 
greater than the coefficient K 2 in the previous bound 
for all channels except the BSC (where the two-level 
quantization does not change the channel). It is also 
interesting to note that for the BEC the new bound 
merges with the improved bound of Sason and Urbanke 
which was shown to be tight even under MPI decoding. 

The requirement of full-rank parity-check matrices 
is not natural when dealing with ensembles of codes; 
from [2], it can be removed by replacing the rate by 
the design rate of the ensemble. 

Numerical Results: Table (T| provides bounds on 
the thresholds under ML decoding of several irregular 
rate i ensembles taken from [20], where the transmis¬ 
sion takes place over the binary input AWGN channel. 
The upper bounds on the achievable rates provide lower 
bounds on the thresholds under ML decoding. The 
bounds shown are the two-level quantization bound of 
Burshtein et al. [2], the four and eight-level quantiza¬ 
tion bounds based on the first approach discussed here, 
which are derived in [26, Corollaries 3.1, 3.2], and the 
un-quantized bound of Theorem 12. II Comparing these 
bounds with the exact thresholds under MPI decoding 
calculated using density evolution (DE) shown in the 
rightmost column, gives an indication of the loss in 
performance due to the sub-optimality of the iterative 
decoder. As mentioned above, comparing the results 
of the quantized bounds with the un-quantized bound 
gives an indication on the effect of quantization at the 
decoder. Table H] shows that the numerical values of 
the bounds which correspond to the optimized eight 
quantization levels are very close to the bounds where 
there is no quantization of the LLR. 


3 Parallel Channels 

In this section we consider LDPC codes transmitted 
over a set of independent parallel MBIOS channels, 
where each code bit is a-priori assigned to one of 
the channels. This setup is useful for modelling a 
wide range of scenarios which include non-uniform 
error protection [16], punctured LDPC codes (see e.g., 
[5], [17]) and LDPC-coded modulation schemes. Non- 
uniform error protection is modelled by assigning the 
bits which require a higher level of protection to a 
channel which is physically degraded w.r.t. the actual 
communication channel. Puncturing is modelled by 
assigning a code bit which is punctured in probability 
p to the channel composed of a BEC with erasure 
probability p cascaded with the actual communication 
channel. In [23] the authors generalized the lower 
bound in Theorem l2.1l for the case of parallel channels. 
It can also be verified that 0 holds for this case. 
Hence, the bounds on the achievable rates, the bit¬ 
error probability and the parity-check density discussed 
in the previous section are also generalized for the 
case of parallel channels. This allows us to study the 
performance of punctured LDPC codes transmitted over 
MBIOS channels. 

4 Punctured LDPC codes 

As discussed in the previous section, punctured codes 
transmitted over a single MBIOS channel can be mod¬ 
elled as codes transmitted over a set of parallel channels 
(see also [16]). The iterative decoder of the punctured 
code utilizes the graph of the original code, where 
the nodes corresponding to the punctured code bits 
receive no channel input and are considered state nodes. 
This allows to use puncturing in order to transmit at 
variable rates using one encoder and one decoder. Some 
fundamental properties of punctured LDPC codes were 
studied in [17]. These results show the high potential 
of puncturing in designing rate-compatible codes which 
operate closely to the channel capacity. 

4.1 Intentionally Punctured LDPC codes 

In [5], Ha and McLaughlin show that good codes can 
be constructed by puncturing good ensembles of LDPC 
codes using a technique called “intentional punctur¬ 
ing”. In this approach, the code bits are partitioned 
into disjoint sets so that each set contains all the 
code bits whose corresponding variable nodes have 
the same degree. The code bits in each one of these 
sets are randomly punctured at a fixed puncturing rate. 
The puncturing pattern is described by a polynomial 
ttq ( x ) = Y^i> 2 TF* 1-1 where 7Tj designates the punctur¬ 
ing rate of code bits of degree i. This forms a particular 
case of transmission over a set of parallel channels, 
where each code bit is assigned to a channel according 
to the number of parity-check equations it is involved 
in. The bound on the achievable rates of ML decoded 








A(z) 

P( a) 

2-Level 
Bound [2] 

4-Level 

Bound 

8-Level 
Bound 

Un-Quantized 
Lower Bound 

DE 

Threshold 

0.38354a; + 0.04237a 2 + 0.57409a 3 

0.24123a 4 + 0.75877a 5 

0.269 dB 

0.370 dB 

0.404 dB 

0.417 dB 

0.809 dB 

0.23802a 4- 0.20997a 2 + 0.03492a 3 + 
0.12015a; 4 + 0.01587a; 6 + 0.00480a; 13 + 
0.37627a; 14 

0.98013a 7 + 0.01987a 8 

0.201 dB 

0.226 dB 

0.236 dB 

0.239 dB 

0.335 dB 

0.21991a: + 0.23328a; 2 + 0.02058a: 3 + 
0.08543a; 5 + 0.06540a; 6 + 0.04767a 7 + 
0.01912a 8 + 0.08064a 18 + 0.22798a 19 

0.64854a 7 +0.34747a 8 + 
0.00399a 9 

0.198 dB 

0.221 dB 

0.229 dB 

0.232 dB 

0.310 dB 

0.19606a + 0.24039a 2 + 0.00228a 5 + 

0.05516a 6 + 0.16602a 7 + 0.04088a 8 + 
0.01064a 9 + 0.00221a 27 + 0.28636a 29 

0.00749a 7 +0.99101a 8 + 
0.00150a 9 

0.194 dB 

0.208 dB 

0.214 dB 

0.216 dB 

0.274 dB 


TABLE I 

Comparison of thresholds for rate one-half ensembles of irregular LDPC codes transmitted over the binary-input 
AWGN channel. The Shannon capacity limit corresponds to Lfc = 0.187dB. The 2-level, 4-level, 8-level and 

A o 

UN-QUANTIZED LOWER BOUNDS ON THE THRESHOLD REFER TO ML DECODING, AND ARE BASED ON [2, THEOREM 2], [26, 

Corollaries 3.1,3.2] and Theorem s. II respectively. The degree distributions of the ensembles and their DE 

THRESHOLDS ARE BASED ON DENSITY EVOLUTION UNDER SUM-PRODUCT DECODING [19], AND ARE TAKEN FROM [20, TABLES 1 AND 2], 


7T U (ai) 

(puncturing pattern) 

Design 

rate 

Capacity 

limit 

Lower bound 
(ML decoding) 

Iterative (IT) 
Decoding 

Fractional gap to 
capacity (ML vs. IT) 

0 

0.500 

0.187 dB 

0.270 dB 

0.393 dB 

> 40.3% 

0.07886a + 0.01405a 2 + 0.06081a 3 + 0.07206a y 

0.528 

0.318 dB 

0.397 dB 

0.526 dB 

> 37.9% 

0.20276a + 0.09305a 2 + 0.03356a 3 + 0.16504a y 

0.592 

0.635 dB 

0.716 dB 

0.857 dB 

> 36.4% 

0.25381a + 0.15000a 2 + 0.34406a 3 + 0.019149a a 

0.629 

0.836 dB 

0.923 dB 

1.068 dB 

> 37.3% 

0.31767a + 0.18079a 2 + 0.05265a 3 + 0.24692a y 

0.671 

1.083 dB 

1.171 dB 

1.330 dB 

> 35.6% 

0.36624a + 0.24119a 2 + 0.49649a 3 + 0.27318a y 

0.719 

1.398 dB 

1.496 dB 

1.664 dB 

> 36.9% 

0.41838a + 0.29462a 2 + 0.05265a 3 + 0.30975a y 

0.774 

1.814 dB 

1.927 dB 

2.115 dB 

> 37.2% 

0.47074a + 0.34447a 2 + 0.02227a 3 + 0.34997a y 

0.838 

2.409 dB 

2.547 dB 

2.781 dB 

> 37.1% 

0.52325a + 0.39074a 2 + 0.01324a 3 + 0.39436a y 

0.912 

3.399 dB 

3.607 dB 

3.992 dB 

> 35.1% 


TABLE II 

Comparison of thresholds for ensembles of intentionally punctured LDPC codes where the original ensemble 
BEFORE PUNCTURING HAS THE DEGREE DISTRIBUTIONS A(a) = 0.25105a + 0.30938a 2 + 0.00104a 3 + 0.43853a: 9 AND 
p(x) = 0.63676a: 6 + 0.36324a: 7 (SO ITS DESIGN RATE IS EQUAL TO i). THE TRANSMISSION TAKES PLACE OVER A BINARY-INPUT AWGN 
CHANNEL. THE TABLE COMPARES VALUES OF Lt REFERRING TO THE CAPACITY LIMIT, THE GENERALIZATION TO PARALLEL CHANNEL OF 
THE BOUND GIVEN IN COROLLARY l2. Il l WHICH PROVIDES A LOWER BOUND ON UNDER ML DECODING), AND THRESHOLDS UNDER 
ITERATIVE MESSAGE-PASSING DECODING. THE FRACTIONAL GAP TO CAPACITY (SEE THE RIGHTMOST COLUMN) MEASURES THE RATIO OF 
THE GAP TO CAPACITY UNDER OPTIMAL ML DECODING AND THE ACHIEVABLE GAP TO CAPACITY UNDER (SUB-OPTIMAL) ITERATIVE 
MESSAGE-PASSING DECODING. THE DEGREE DISTRIBUTIONS FOR THE ENSEMBLE OF LDPC CODES, AND THE POLYNOMIALS WHICH 
CORRESPOND TO ITS PUNCTURING PATTERNS ARE TAKEN FROM [5, TABLE 2], 


LDPC codes transmitted over parallel channels, can 
be used to find lower bounds on the thresholds which 
intentionally punctured LDPC codes achieve under ML 
decoding. Comparing these bounds with exact thresh¬ 
olds calculated in [5] using DE, gives an indication as 
to the inherent loss caused by the sub-optimality of the 
iterative decoder for various puncturing patterns and 
rates. The bound also gives an indication of the loss of 
punctured codes even under ML decoding due to the 
structure of the codes (as compared to random coding). 
Table [H] presents the lower bound on the threshold 
under ML decoding of a sequence of ensembles of 
LDPC codes with various puncturing rates, where the 
transmission takes place over the binary input AWGN 
channel. It can be seen that for all the puncturing rates 
shown in Table mi more than of the gap to capacity 
is attributed to the structure of the codes, while the rest 
is due to the sub-optimality of the iterative decoder. 


4.2 Randomly punctured LDPC codes 

A second approach to the construction of ensembles 
of punctured LDPC codes is to a-priori select a subset 
of the code bits and randomly puncture these bits at 
a given rate. This method can also be modelled as a 
specific case of transmission over a set of two parallel 
channels. By observing the lower bound on the parity- 
check density for parallel channels, one is able to derive 
a lower bound on the decoding complexity per iteration 
of randomly punctured LDPC codes. 

In [14], Pfister et al. provided explicit construc¬ 
tions of ensembles of irregular repeat-accumulate (IRA) 
codes which asymptotically (as the block length tends 
to infinity) achieve the capacity of the BEC under 
MPI decoding with bounded encoding and decoding 
complexities per information bit. This result came 
as a surprise in light of parallel results which show 
that for ensembles of LDPC codes without puncturing 






























and systematic IRA codes, their decoding complexity 
per information bit should become unbounded as the 
gap to capacity vanishes (see [21] and [22]). Since 
the convergence rate (in terms of the block length) 
of the codes in [14] was rather slow, these codes 
were not record breaking for short to moderate block 
lengths. Several constructions of ensembles of system¬ 
atic accumulate-repeat-accumulate (ARA) codes were 
recently introduced by Pfister and Sason [15]. These 
ensembles achieve capacity of the BEC under MPI 
decoding with bounded complexity per information bit, 
and outperform other capacity-achieving ensembles of 
codes on graphs, even for moderate block lengths. In 
[7], Hsu and Anastasopoulos introduced ensembles of 
codes which achieve the capacity of general MBIOS 
channels under ML decoding with bounded number of 
edges in the Tanner graph per information bit, and for 
the BEC, they also achieve bounded complexity per in¬ 
formation bit under MPI decoding. All of the capacity- 
achieving ensembles on the BEC with bounded com¬ 
plexity rely on puncturing, as could be expected from 
the information-theoretic bounds for codes on graphs 
in [13], [14, Theorems 3 and 4] and [23], 
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